Guidelines for Reporting Trial Protocols and Completed Trials Modified Due to the COVID-19 Pandemic and Other Extenuating Circumstances: The CONSERVE 2021 Statement.

Importance: Extenuating circumstances can trigger unplanned changes to randomized trials and introduce methodological, ethical, feasibility, and analytical challenges that can potentially compromise the validity of findings. Numerous randomized trials have required changes in response to the COVID-19 pandemic, but guidance for reporting such modifications is incomplete. Objective: As a joint extension for the CONSORT and SPIRIT reporting guidelines, CONSERVE (CONSORT and SPIRIT Extension for RCTs Revised in Extenuating Circumstances) aims to improve reporting of trial protocols and completed trials that undergo important modifications in response to extenuating circumstances. Evidence: A panel of 37 international trial investigators, patient representatives, methodologists and statisticians, ethicists, funders, regulators, and journal editors convened to develop the guideline. The panel developed CONSERVE following an accelerated, iterative process between June 2020 and February 2021 involving (1) a rapid literature review of multiple databases (OVID Medline, OVID EMBASE, and EBSCO CINAHL) and gray literature sources from 2003 to March 2021; (2) consensus-based panelist meetings using a modified Delphi process and surveys; and (3) a global survey of trial stakeholders. Findings: The rapid review yielded 41 673 citations, of which 38 titles were relevant, including emerging guidance from regulatory and funding agencies for managing the effects of the COVID-19 pandemic on trials. However, no generalizable guidance for all circumstances in which trials and trial protocols might face unanticipated modifications were identified. The CONSERVE panel used these findings to develop a consensus reporting guidelines following 4 rounds of meetings and surveys. Responses were received from 198 professionals from 34 countries, of whom 90% (n = 178) indicated that they understood the concept definitions and 85.4% (n = 169) indicated that they understood and could use the implementation tool. Feedback from survey respondents was used to finalize the guideline and confirm that the guideline's core concepts were applicable and had utility for the trial community. CONSERVE incorporates an implementation tool and checklists tailored to trial reports and trial protocols for which extenuating circumstances have resulted in important modifications to the intended study procedures. The checklists include 4 sections capturing extenuating circumstances, important modifications, responsible parties, and interim data analyses. Conclusions and Relevance: CONSERVE offers an extension to CONSORT and SPIRIT that could improve the transparency, quality, and completeness of reporting important modifications to trials in extenuating circumstances such as COVID-19.

Development, implementation and evaluation of an online course on evidence-based healthcare for consumers.

BACKGROUND: Evidence-based healthcare (EBHC) principles are essential knowledge for patient and consumer ("consumer") engagement as research and research implementation stakeholders. The aim of this study was to assess whether participation in a free, self-paced online course affects confidence in explaining EBHC topics. The course comprises six modules and evaluations which together take about 6 h to complete. METHODS: Consumers United for Evidence-based Healthcare (CUE) designed, tested and implemented a free, online course for consumers, Understanding Evidence-based Healthcare: A Foundation for Action ("Understanding EBHC"). The course is offered through the Johns Hopkins Bloomberg School of Public Health. Participants rated their confidence in explaining EBHC topics on a scale of 1 (lowest) to 5 (highest), using an online evaluation provided before accessing the course ("Before") and after ("After") completing all six course modules. We analyzed data from those who registered for the course from May 31, 2007 to December 31, 2018 (n = 15,606), and among those persons, the 11,522 who completed the "Before" evaluation and 4899 who completed the "After" evaluation. Our primary outcome was the overall mean of within-person change ("overall mean change") in self-reported confidence levels on EBHC-related topics between "Before" and "After" evaluations among course completers. Our secondary outcomes were the mean within-person change for each of the 11 topics (mean change by topic). RESULTS: From May 31, 2007 to December 31, 2018, 15,606 individuals registered for the course: 11,522 completed the "Before" evaluation, and 4899 of these completed the "After" evaluation (i.e., completed the course). The overall mean change in self-reported confidence levels (ranging from 1 to 5) from the "Before" to "After" evaluation was 1.27 (95% CI, 1.24-1.30). The mean change by topic ranged from 1.00 (95% CI, 0.96-1.03) to 1.90 (95% CI, 1.87-1.94). CONCLUSION: Those who seek to involve consumer stakeholders can offer Understanding EBHC as a step toward meaningful consumer engagement. Future research should focus on long-term impact assessment of online course such as ours to understand whether confidence is retained post-course and applied appropriately.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study.

BACKGROUND: There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses. METHODS: We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome. RESULTS: We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews). CONCLUSIONS: Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Evaluation of Systematic Reviews of Interventions for Retina and Vitreous Conditions.

Importance: Patient care and clinical practice guidelines should be informed by evidence from reliable systematic reviews. The reliability of systematic reviews related to forthcoming guidelines for retina and vitreous conditions is unknown. Objectives: To summarize the reliability of systematic reviews on interventions for 7 retina and vitreous conditions, describe characteristics of reliable and unreliable systematic reviews, and examine the primary area in which they appeared to be lacking. Design, Setting, and Participants: A cross-sectional study of systematic reviews was conducted. Systematic reviews of interventions for retina- and vitreous-related conditions in a database maintained by the Cochrane Eyes and Vision United States Satellite were identified. Databases that the reviewers searched, whether any date or language restrictions were applied, and bibliographic information, such as year and journal of publication, were documented. The initial search was conducted in March 2007, and the final update was performed in July 2018. The conditions of interest were age-related macular degeneration; diabetic retinopathy; idiopathic epiretinal membrane and vitreomacular traction; idiopathic macular hole; posterior vitreous detachment, retinal breaks, and lattice degeneration; retinal and ophthalmic artery occlusions; and retinal vein occlusions. The reliability of each review was evaluated using prespecified criteria. Data were extracted by 2 research assistants working independently, with disagreements resolved through discussion or by 1 research assistant with verification by a senior team member. Main Outcomes and Measures: Proportion of reviews that meet all of the following criteria: (1) defined eligibility criteria for study selection, (2) described conducting a comprehensive literature search, (3) reported assessing risk of bias in included studies, (4) described using appropriate methods for any meta-analysis performed, and (5) provided conclusions consistent with review findings. Results: A total of 327 systematic reviews that addressed retina and vitreous conditions were identified; of these, 131 reviews (40.1%) were classified as reliable and 196 reviews (59.9%) were classified as not reliable. At least 1 reliable review was found for each of the 7 retina and vitreous conditions. The most common reason that a review was classified as not reliable was lack of evidence that a comprehensive literature search for relevant studies had been conducted (149 of 196 reviews [76.0%]). Conclusion and Relevance: The findings of this study suggest that most systematic reviews that addressed interventions for retina and vitreous conditions were not reliable. Systematic review teams and guideline developers should work with information professionals who can help navigate sophisticated and varied syntaxes required to search different resources.

Opportunities for selective reporting of harms in randomized clinical trials: Selection criteria for non-systematic adverse events.

BACKGROUND: Adverse events (AEs) in clinical trials may be reported in multiple sources. Different methods for reporting adverse events across trials or across sources for a single trial may produce inconsistent information about the adverse events associated with interventions. METHODS: We compared the methods authors use to decide which AEs to include in a particular source (i.e., "selection criteria"), including the number of different types of AEs reported (i.e., rather than the number of events). We compared sources (e.g., journal articles, clinical study reports (CSRs)) of trials for two drug-indications-gabapentin for neuropathic pain and quetiapine for bipolar depression. Electronic searches were completed in 2015. We identified selection criteria and assessed how criteria affected AE reporting. RESULTS: We identified 21 gabapentin and 7 quetiapine trials. We found 6 gabapentin CSRs and 2 quetiapine CSRs, all written by drug manufacturers. All CSRs reported all AEs without applying selection criteria; by comparison, no other source reported all AEs, and 15/68 (22%) gabapentin sources and 19/48 (40%) quetiapine sources reported using selection criteria. Selection criteria greatly affected the number of AEs reported. For example, 67/316 (21%) AEs in one quetiapine trial met the criterion "occurring in ≥2% of participants in any treatment group," while only 5/316 (2%) AEs met the criterion "occurring in ≥10% of quetiapine-treated patients and twice as frequent in the quetiapine group as the placebo group." CONCLUSIONS: Selection criteria for reporting AEs vary across trials and across sources for individual trials. If investigators do not pre-specify selection criteria, they might "cherry-pick" AEs based on results. Even if investigators pre-specify selection criteria, selective reporting will produce biased meta-analyses and clinical practice guidelines. Data about all AEs identified in clinical trials should be publicly available; however, sharing data will not solve all the problems identified in this study.

A randomized trial provided new evidence on the accuracy and efficiency of traditional vs. electronically annotated abstraction approaches in systematic reviews.

OBJECTIVES: Data Abstraction Assistant (DAA) is a software for linking items abstracted into a data collection form for a systematic review to their locations in a study report. We conducted a randomized cross-over trial that compared DAA-facilitated single-data abstraction plus verification ("DAA verification"), single data abstraction plus verification ("regular verification"), and independent dual data abstraction plus adjudication ("independent abstraction"). STUDY DESIGN AND SETTING: This study is an online randomized cross-over trial with 26 pairs of data abstractors. Each pair abstracted data from six articles, two per approach. Outcomes were the proportion of errors and time taken. RESULTS: Overall proportion of errors was 17% for DAA verification, 16% for regular verification, and 15% for independent abstraction. DAA verification was associated with higher odds of errors when compared with regular verification (adjusted odds ratio [OR] = 1.08; 95% confidence interval [CI]: 0.99-1.17) or independent abstraction (adjusted OR = 1.12; 95% CI: 1.03-1.22). For each article, DAA verification took 20 minutes (95% CI: 1-40) longer than regular verification, but 46 minutes (95% CI: 26 to 66) shorter than independent abstraction. CONCLUSION: Independent abstraction may only be necessary for complex data items. DAA provides an audit trail that is crucial for reproducible research.

Reliability of the Evidence Addressing Treatment of Corneal Diseases: A Summary of Systematic Reviews.

Importance: Patient care should be informed by clinical practice guidelines, which in turn should be informed by evidence from reliable systematic reviews. The American Academy of Ophthalmology is updating its Preferred Practice Patterns (PPPs) for the management of the following 6 corneal diseases: bacterial keratitis, blepharitis, conjunctivitis, corneal ectasia, corneal edema and opacification, and dry eye syndrome. Objective: To summarize the reliability of the existing systematic reviews addressing interventions for corneal diseases. Data Source: The Cochrane Eyes and Vision US Satellite database. Study Selection: In this study of published systematic reviews from 1997 to 2017 (median, 2014), the Cochrane Eyes and Vision US Satellite database was searched for systematic reviews evaluating interventions for the management of any corneal disease, combining eyes and vision keywords and controlled vocabulary terms with a validated search filter. Data Extraction and Synthesis: The study classified systematic reviews as reliable when each of the following 5 criteria were met: the systematic review specified eligibility criteria for inclusion of studies, conducted a comprehensive literature search for studies, assessed risk of bias of the individual included studies, used appropriate methods for quantitative syntheses (meta-analysis) (only assessed if meta-analysis was performed), and had conclusions that were supported by the results of the systematic review. They were classified as unreliable if at least 1 criterion was not met. Main Outcomes and Measures: The proportion of systematic reviews that were reliable and the reasons for unreliability. Results: This study identified 98 systematic reviews that addressed interventions for 15 corneal diseases. Thirty-three of 98 systematic reviews (34%) were classified as unreliable. The most frequent reasons for unreliability were that the systematic review did not conduct a comprehensive literature search for studies (22 of 33 [67%]), did not assess risk of bias of the individual included studies (13 of 33 [39%]), and did not use appropriate methods for quantitative syntheses (meta-analysis) (12 of 17 systematic reviews that conducted a quantitative synthesis [71%]). Sixty-five of 98 systematic reviews (66%) were classified as reliable. Forty-two of the 65 reliable systematic reviews (65%) addressed corneal diseases relevant to the 2018 American Academy of Ophthalmology PPPs; 33 of these 42 systematic reviews (79%) are cited in the 2018 PPPs. Conclusions and Relevance: One in 3 systematic reviews addressing interventions for corneal diseases are unreliable and thus were not used to inform PPP recommendations. Careful adherence by systematic reviewers and journal editors to well-established best practices regarding systematic review conduct and reporting might help make future systematic reviews in eyes and vision more reliable.

Harms are assessed inconsistently and reported inadequately part 1: systematic adverse events.

OBJECTIVES: We examined systematic adverse events (AEs) in Part 1 (of 2) of a study describing the assessment and reporting of AEs in clinical trials. STUDY DESIGN AND SETTING: We examined 52 public and nonpublic data sources about trials of quetiapine for bipolar depression using data from the Multiple Data Sources study. We extracted and compared information about systematic AEs (i.e., AEs assessed for all participants) in six prespecified domains: cardiovascular, cholesterol, endocrine, extrapyramidal symptoms, mania, and weight. RESULTS: Eligible trials did not assess and report the same systematic AEs, and most results were not available in public sources. Overall, public sources reported 159 results, of which 92 of 159 (58%) included sufficient statistical information to calculate the treatment effect and its precision. Nonpublic sources reported 636 results; 630 of 636 (99%) reported sufficient statistical information. CONCLUSION: Systematic AEs were defined and analyzed in many ways, which led to many numerical results. Most systematic AEs were not mentioned in public sources. To minimize bias, methods for defining and analyzing potential AEs should be prespecified in trial registers and protocols. All trial results should be publicly available so that stakeholders can compare benefits and AEs. Trials could report core sets of AEs to facilitate decision-making.

Harms are assessed inconsistently and reported inadequately Part 2: nonsystematic adverse events.

OBJECTIVE: We examined nonsystematic adverse events (AEs) in Part 2 (of 2) of a study describing the assessment and reporting AEs in clinical trials. STUDY DESIGN AND SETTING: We examined 21 trials of gabapentin for neuropathic pain (52 sources) and seven trials of quetiapine for bipolar depression (80 sources) using data from the Multiple Data Sources study. We extracted and compared information about nonsystematic AEs (i.e., AEs that were not assessed for every participant), including AEs categorized as "serious." We recorded whether AEs were grouped by anatomic or physiological system. RESULTS: Trials of the same drug reported information about different AEs. Information in public sources was inadequate for decision-making. No public source reported all AEs, or all serious AEs, identified in nonpublic sources about the same trial. Of trials with only public sources, 2/15 (13%) gabapentin and 0/3 (0%) quetiapine trials grouped AEs by anatomic or physiological system. CONCLUSION: Public sources contained little information about nonsystematic AEs, including serious AEs. Grouping might make nonsystematic AEs easier to detect; however, most public sources did not report grouped AEs. Standards are needed to improve the collection and reporting of nonsystematic AEs so that stakeholders can use trials to assess the balance of potential benefits and harms.

Gender and Editorial Authorship in High-Impact Epidemiology Journals.

Women comprise about half of senior epidemiologists, but little is known about whether they are also viewed as leaders (i.e., authorities) in the field. We believe editorial roles are markers of leadership in a field. Our objective was to describe the distribution of gender across authorship of editorials published in 5 high-impact epidemiology journals over the past 8 years. We included editorials and commentaries published in American Journal of Epidemiology, European Journal of Epidemiology, Epidemiology, International Journal of Epidemiology, and Journal of Clinical Epidemiology between 2010 and 2017. We classified genders of all authors as woman, man, or unknown and computed the proportions of women editorial authors over all journals and according to position (e.g., first author). Only 31% (682/2,228) of all editorial authors and 36% (524/1,477) of unique editorial authors (i.e., counting each editorial author name only once) were women. We identified 1,180 editorials; 594 had sole authors, 24% (141/594) of whom were women, and 586 had 2 or more authors, 31% (184/586) of which had women as first authors. If women are underrepresented as editorial authors across epidemiology journals (e.g., as a marker of epidemiology leadership), the situation merits immediate correction.

Caveat emptor: the combined effects of multiplicity and selective reporting.

Clinical trials and systematic reviews of clinical trials inform healthcare decisions. There is growing concern, however, about results from clinical trials that cannot be reproduced. Reasons for nonreproducibility include that outcomes are defined in multiple ways, results can be obtained using multiple methods of analysis, and trial findings are reported in multiple sources ("multiplicity"). Multiplicity combined with selective reporting can influence dissemination of trial findings and decision-making. In particular, users of evidence might be misled by exposure to selected sources and overly optimistic representations of intervention effects. In this commentary, drawing from our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study and evidence from previous research, we offer practical recommendations to enhance the reproducibility of clinical trials and systematic reviews.

Evaluation of Clinical Questions and Patient-Important Outcomes Associated With the Treatment of Age-Related Macular Degeneration.

Importance: Identifying and prioritizing unanswered clinical questions may help to best allocate limited resources for research associated with the treatment of age-related macular degeneration (AMD). Objective: To identify and prioritize clinical questions and outcomes for research associated with the treatment of AMD through engagement with professional and patient stakeholders. Design, Setting, and Participants: Multiple cross-sectional survey questions were used in a modified Delphi process for panel members of US and international organizations, the American Academy of Ophthalmology (AAO) Retina/Vitreous Panel (n=7), health care professionals from the American Society of Retinal Specialists (ASRS) (n=90), Atlantic Coast Retina Conference (ACRC) and Macula 2017 meeting (n=34); and patients from MD (Macular Degeneration) Support (n=46). Data were collected from January 20, 2015, to January 9, 2017. Main Outcomes and Measures: The prioritizing of clinical questions and patient-important outcomes for AMD. Results: Seventy clinical questions were derived from the AAO Preferred Practice Patterns for AMD and suggestions by the AAO Retina/Vitreous Panel. The AAO Retina/Vitreous Panel assessed all 70 clinical questions and rated 17 of 70 questions (24%) as highly important. Health care professionals assessed the 17 highly important clinical questions and rated 12 of 17 questions (71%) as high priority for research to answer; 9 of 12 high-priority clinical questions were associated with aspects of anti-vascular endothelial growth factor agents. Patients assessed the 17 highly important clinical questions and rated all as high priority. Additionally, patients identified 6 of 33 outcomes (18%) as most important to them (choroidal neovascularization, development of advanced AMD, retinal hemorrhage, gain of vision, slowing vision loss, and serious ocular events). Conclusions and Relevance: Input from 4 stakeholder groups suggests good agreement on which 12 priority clinical questions can be used to underpin research related to the treatment of AMD. The 6 most important outcomes identified by patients were balanced between intended effects of AMD treatment (eg, slowing vision loss) and adverse events. Consideration of these patient-important outcomes may help to guide clinical care and future areas of research.

Research Questions and Outcomes Prioritized by Patients With Dry Eye.

Importance: Dry eye is a common ocular surface condition with significant influence on patient quality of life and societal economic burden. There is an urgent need to prioritize new research for dry eye. Objective: To identify and rank research questions and outcomes important to patients with dry eye. Design, Setting, and Participants: This study was conducted using the following 6 steps: (1) identifying research questions from a previous survey of clinicians who treat patients with dry eye; (2) identifying outcomes from existing research (systematic reviews and their cited clinical trials in the Cochrane Eyes and Vision US Satellite database of eyes and vision reviews, and National Eye Institute-funded clinical trials registered on ClinicalTrials.gov) as of June 13, 2017; (3) identifying a sample of patients with dry eye from the email subscribers to the online newsletter KeratoScoop; (4) and (5) conducting a 2-round Delphi survey of those patients online in November and December 2017, respectively; and (6) designating and ranking questions and outcomes as important. Main Outcomes and Measures: Importance assigned to research questions and outcomes for dry eye. A research question or outcome ranked by at least 75% of patients as 6 or higher on a scale of 0 to 10 was considered important. Results: Among the 420 patients from 15 countries who completed both rounds of the Delphi survey, most were 60 years of age or older (233 [56%]), female (348 [83%]), white (393 [94%]), and of non-Hispanic ethnicity (398 [95%]). Among the 12 questions that clinicians had previously prioritized, patients rated 8 as important. The top 3 questions pertained to effectiveness of patient education, environmental modifications, and topical anti-inflammatory eye drops for dry eye. Among the 109 outcomes identified in existing research on dry eye, patients rated 26 as important. Ten of these 26 were unpopular in existing research, with fewer than 10% of 158 studies reporting these outcomes. Of the 10 most important outcomes, 9 were associated with symptoms or quality of life. The 3 outcomes rated most important by patients were ocular burning or stinging, ocular discomfort, and ocular pain. Conclusions and Relevance: This study identified research questions and outcomes important to patients with dry eye. A considerable gap was noted between outcomes in existing research on dry eye and outcomes patients consider important. Future research on dry eye should consider addressing the important research questions and outcomes identified herein, taking into account the patient perspective.

Methods to identify and prioritize patient-centered outcomes for use in comparative effectiveness research.

BACKGROUND: We used various methods for identifying and prioritizing patient-centered outcomes (PCOs) for comparative effectiveness research (CER). METHODS: We considered potential PCOs ("benefits" and "harms") related to (1) gabapentin for neuropathic pain and (2) quetiapine for bipolar depression. Part 1 (April 2014 to March 2015): we searched for PCO research and core outcome sets (COSs). We conducted electronic searches of bibliographic databases and key websites and examined FDA prescribing information and reports of clinical trials and systematic reviews. We asked patient and clinician co-investigators to identify PCOs. Part 2 (not part of our original study protocol): in 2015, we surveyed members of The TMJ Association, Ltd., a patient group associated with temporomandibular disorders (4130 invitations sent). Participants prioritized (1) the importance of six potential benefits and (2) 21 potential harms selected by the investigators in part 1, using stated preference methods. We calculated descriptive statistics. RESULTS: In part 1, we identified a COS for pain, the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) recommendations. The COS identified several important benefits, but it lacked specific recommendations about which potential harms to include in CER. We did not identify a COS for bipolar depression. Research reports, prescribing information, and patient co-investigators helped identify but not prioritize outcomes. We abandoned our electronic search for PCO research because we found it would be resource-intensive and yield few relevant reports. In part 2, surveying patients was useful for prioritizing PCOs. Members of The TMJ Association, Ltd., completed the survey (N = 746) and successfully prioritized both benefits and harms. Participants did not identify many benefits other than those we identified in part 1; several participants identified additional harms. CONCLUSIONS: These exploratory results could inform future research about identifying and prioritizing PCOs. We found that stakeholder co-investigators and research reports contributed to identifying PCOs; surveying a patient group contributed to prioritizing PCOs. Prioritizing potential harms was particularly challenging because there are many more potential harms than potential benefits. Methods for identifying and prioritizing potential benefits for CER might not be appropriate for harms. Further research is needed to determine the generalizability of these results.

Standards for design and measurement would make clinical research reproducible and usable.

We find standards useful in everyday life and in science, although we do not always follow them. Adopting new standards can be expensive, so there may be a strong incentive to maintain the status quo rather than adopt new standards. The scientific community has many standards encompassing both doing clinical research and reporting it, including standards for design and measurement. Although existing research standards have improved both research and its reporting, we need to unify existing standards and to fill the gaps between steps throughout the research process. Existing gaps include implementation of standards and links between standards for study registration (to know about all studies undertaken), study protocols (to identify the preplanned study design and methods), data collection (to assess outcomes that are important and comparable across studies), dissemination of findings (to know the results of previous studies), data sharing (to make best use of existing data), and evidence synthesis (to draw appropriate conclusions from the body of evidence). The scientific community must work together to harmonize existing standards, to ensure that standards are kept up to date, to check that standards are followed, and to develop standards where they are still needed. A unified system of standards will make our work more reproducible.

Correction to: Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol.

The correct title of the article [1] should be "Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol".

Correction to: Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic review.

CORRECTION: The correct title of the article [1] should be "Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol". The article is a protocol for a methodological study, not a systematic review.

Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study).

Data for individual trials included in systematic reviews may be available in multiple sources. For example, a single trial might be reported in 2 journal articles and 3 conference abstracts. Because of differences across sources, source selection can influence the results of systematic reviews. We used our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study, and evidence from previous studies, to develop practical guidance for using multiple data sources in systematic reviews. We recommend the following: (1) Specify which sources you will use. Before beginning a systematic review, consider which sources are likely to contain the most useful data. Try to identify all relevant reports and to extract information from the most reliable sources. (2) Link individual trials with multiple sources. Write to authors to determine which sources are likely related to the same trials. Use a modified Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flowchart to document both the selection of trials and the selection of sources. (3) Follow a prespecified protocol for extracting trial characteristics from multiple sources. Identify differences among sources, and contact study authors to resolve differences if possible. (4) Prespecify outcomes and results to examine in the review and meta-analysis. In your protocol, describe how you will handle multiple outcomes within each domain of interest. Look for outcomes using all eligible sources. (5) Identify which data sources were included in the review. Consider whether the results might have been influenced by data sources used. (6) To reduce bias, and to reduce research waste, share the data used in your review.